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STATEMENT OF MATERIAL FACTS 1 

A. The Plaintiff Sponsoring Organizations 

1. Plaintiffs, the Sponsoring Organizations, are each District of Columbia not-for- 
profit corporations (Levine Decl., ]f 4; Ernesto Decl., ]f 3; Wise Decl., ]f 3). Each is an active 
professional organization that engages in many activities for many purposes other than to create 
the Standards for Educational and Psychological Testing at issue in this case. 

2. American Education Resource Association (AERA) is the major national 
scientific society for research on education and learning. AERA’s mission is to advance 
knowledge about education, to encourage scholarly inquiry related to education, and to promote 
the use of research to improve education and serve the public good (Levine Decl., ]j 5). 

3. The American Psychological Association (APA) is the largest scientific and 
professional organization representing psychology in the United States. APA is the world’s 
largest association of psychologists and counts a vast number of researchers, educators, 
clinicians, consultants and students among its members. APA’s mission is to advance the 
creation, communication, and application of psychological knowledge to benefit society and 
improve people’s lives (Ernesto Deck, ]f 4). 

4. The National Council on Measurement in Education (NCME) is a professional 
organization for individuals involved in assessment, evaluation, testing, and other aspects of 
educational measurement. NCME’s members are involved in the construction and use of 
standardized tests; new forms of assessment, including performance- based assessment; program 
design; and program evaluation (Wise Deck, ]f 4). 

1 Much of the supporting material cited in this Statement was provided previously in connection 
with the earlier summary judgment briefing. It is tendered again, for the convenience of the 
Court. In a number of instances, only deposition excerpts are provided. 
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B. Creation of the Standards for Educational and Psychological Testing 

5. The Sponsoring Organizations have been preparing and publishing versions of the 
Standards for over fifty years. In 1954, APA prepared and published the “Technical 
Recommendations for Psychological Tests and Diagnostic Techniques” (Camara Decl., ]J 7; 
Ernesto Decl, 1} 5). In 1955, AERA and NCME prepared and published a companion document 
titled, “Technical Recommendations for Achievement Tests” (Levine Decl., U 6; Camara Decl., ]f 
7; Wise Decl., ]f 5). Subsequently, a joint committee of the three organizations modified, revised, 
and consolidated the two documents into the first Joint Standards. Beginning with the 1966 
revision, the three organizations collaborated in developing the “Joint Standards” (or simply, the 
“Standards”). Each subsequent revision of the Standards has been careful to note that it is a 
revision and update of the prior version (Levine Decl., ]f 6; Camara Decl., ]j 7; Ernesto Decl., ]f 6; 
Wise Decl., K 6). 

6. Specifically, beginning in the mid-1950s, Plaintiffs formed and periodically 
reconstituted a committee of experienced experts in psychological and educational assessment, 
charged with the initial development of the Technical Recommendations and then each 
subsequent revision of the (renamed) Standards. These committees were fonned by the 
Plaintiffs’ presidents (or designees), who would meet and agree on the committees’ membership. 
Beginning with the 1966 version of the Standards, this committee has been called the “Joint 
Committee” (Levine Decl., ]f 7; Camara Decl., ]| 8; Ernesto Decl., ]f 7; Wise Decl., ]f 7). 

7. Financial and operational oversight for revising, promoting, distributing, and 
selling the 1999 and 2014 Standards has been undertaken by a periodically reconstituted 
“Management Committee,” comprised of designees of the three Plaintiffs (Levine Deck, ]f 8; 
Camara Deck, ]f 9; Schneider Deck, ]f 4; Ernesto Deck, ]f 8; Wise Deck, ]| 8). All members of the 
Joint Committee(s) and the Management Committee(s) are unpaid volunteers. The expenses 
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associated with the ongoing development and publication of the Standards include travel and 
lodging expenses (for the Joint Committee and Management Committee members), support staff 
time, printing and shipment of bound volumes, and advertising costs (Levine Decl., ]J 9; Camara 
Decl., ]f 10; Schneider Decl, ]j 5; Ernesto Decl, ]f 9; Wise Decl., f 9). 

8. Many different fields of endeavor rely on assessments, and Plaintiffs seek to 
ensure that the range of these fields of endeavor is represented in the Joint Committees’ 
membership — e.g., admissions, achievement, clinical counseling, educational, licensing- 
credentialing, employment, policy, and program evaluation. Similarly, the Joint Committee’s 
members represent expertise across major functional assessment areas, including validity, 
equating, reliability, test development, scoring, reporting, interpretation, and large scale 
interpolation (Levine Deck, U 10; Ernesto Deck, ]j 10; Wise Deck, f 10). 

9. AERA currently serves as publisher of the Standards, which are made available 
for purchase through the AERA website. Levine Tr. 29, 55. AERA is a small organization, and a 
purchaser may ordinarily contact some in person at AERA if they are having difficulty placing 
an order. Levine Tr. 61-62. Sales are made to individual entities, libraries across the country, 
college bookstores and academic institutions, with spikes in sales sometimes associate with the 
academic calendar. See Levine Tr. 49-50. AERA sells the volumes for hard copy purchase, and 
one can also purchase an electronic (read only) version of the 2014 Standards - or one can make 
a combined purchase of both the electronic and hard copy for a discounted price. Levine Tr. 52- 
54, 61,Exh. 1308, 1309. 

10. Plaintiffs promote and sell copies of the Standards via a variety of referrals to the 
AERA website, at annual meetings, in public offerings to students, and to educational institution 
faculty. Advertisements promoting the Standards have appeared in meeting brochures, in 
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scholarly journals, and in the hallways at professional meetings (Levine Decl., ]f 14, Exh. NNN; 
Ernesto Decl, H 28, Exh. UU; Wise Decl., f 21, Exh. KECK). 

11. Distribution of the Standards is monitored by the Sponsoring Organizations. 
AERA, the now-designated publisher of the Standards, sometimes does provide promotional 
complementary print copies to students or professors. Except for these few complementary print 
copies, however, the Standards are not given away for free; they are not made available to the 
public by any of the three organizations, or given to anyone to copy free of charge (Levine Decl., 
]f 16; Ernesto Decl., U 29; Wise Decl., K 22). To date, Plaintiffs have never posted, or authorized 
the posting of, a digitized copy of the 1999 Standards on any publicly accessible website (Levine 
Decl., f 16; Ernesto Decl., ]j 30; Wise Decl., f 23). Thus, the Sponsoring Organizations do not 
make the Standards accessible (except for purchase) on-line in read only format. (C.f. American 
Society for Testing and Materials, et al. v. Public.Resource.Org, Inc., 896 F.3d 437, 453 (2018) 
(“the SDOs, by their own admission, make copies of their standards freely available online in 
controlled reading rooms”)). Doing so would logically reduce AERA’s ability to sell the 
Standards, and thus reduce revenue for the overall enterprise of maintaining the Standards. 

12. The 1999 Standards have historically been sold at modest retail prices ranging 
from $25.95 to $49.95 per copy. (Levine Deck, ]j 17). Current pricing for members for either a 
hard copy or e-book (single user) of the 2014 edition is $49.95, or as a bundle (hard copy and 
single user e-book) at $59.95. The non-member prices for either a hard copy or single user e- 
book is $69.95, with $79.95 for the bundle. The 1999 edition is available in hardcopy at $35.95 
for members and $45.95 for non-members, but the advertisement/offer of the 1999 Standards 
expressly sets forth the caution that the volume has been superseded. Levine Tr. Exh. 1308, 
1309. 
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13. The 1999 Standards were first offered for sale in mid-1999. Sales in the year 
2000 were estimated at 3797; for 2001, at 3755. They were 5592 in 2002; 3310 in 2003; 3218 in 
in 2004; 3803 in 2005; 3888 in 2006; 3077 in 2007; 3358 in 2008; 2590 in 2009; 3043 in 2010; 
2132 in 2011; 1649 in 2012; 1732 in 2013; and 855 in 2014. Levine Tr. Exh. 1207. Declining 
sales 2011-2014 are coincident with, and might be attributed to, a combination of factors 
including: PRO’S uploading of the 1999 Standards on-line beginning in May 2012 and running 
through June 2014; saturation of parts of the market; or anticipation of the impending release of a 
new edition, the 2014 Standards, which was introduced in the middle of 2014. 

14. After the 2014 version of the Standards was published in late summer of 2014, 
AERA for a time discontinued sales of the 1999 Standards in order to encourage sales of the 
newly-revised edition — the 2014 Standards (Levine Deck, ]f 19, Exh. PPP). However, so long 
as purchasers are aware that it is no longer the current edition - which is made clear on the 
AERA website - the 1999 Standards continue to have value for those in the testing and 
assessment profession who (i) need to know the state of best testing practices as they existed 
between 1999 and 2014, (ii) believe they still may be held accountable to the guidance of the 
1999 Standards, or (iii) study changes in best testing and assessment practices over time. In the 
summer of 2015, AERA resumed sales of the 1999 Standards (Levine Deck, ]J 20, Exh. QQQ). 

15. The production and updating of the Standards is financed solely through sales 
revenues. All revenue from the sale of the 1999 Standards above expenses is used to cover the 
publishing costs of the Standards and for the preparation of subsequent editions of the Standards. 
The Sponsoring Organizations do not distribute any proceeds from the sales of the Standards to 
the Sponsoring Organizations. Thus, generation of revenue allows the Sponsoring Organizations 
to develop up-to-date, high quality Standards that otherwise would not be developed due to the 
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time and effort that goes into producing them (Levine Decl., f 21; Geisinger Decl, ]j 22; Camara 
Decl., ]f 19; Ernesto Decl., ]j 31). Without income from the sales of the Standards to offset 
production costs and to allow for further revisions, it is very likely that the Sponsoring 
Organizations would no longer undertake to periodically update them (Levine Deck, ]j 22; 
Ernesto Deck, ]f 32; Wise Deck, K 24; Geisinger Deck, ][22). 

16. Plaintiffs at one time considered soliciting funding for the revision process from 
third party sources like governmental agencies, foundations, and other associations interested in 
testing and assessment issues. But Plaintiffs rejected this option due to the difficulty of procuring 
the funding as well as potential conflicts of interest that could arise from such a system. The 
Sponsoring Organizations therefore concluded that revisions should be self-funding — that is, 
from sale of prior editions of the Standards (Levine Deck, ]j 23; Camara Deck, ]f 20). 

17. The Sponsoring Organizations do not, so far as the record shows, market any 
associated services for sale, such as seminars, as a way to generate revenue or profit through 
collateral businesses or related products. (Compare 896 F.3d at 453 (“[C]an the SDOs continue 
to make money on derivative good such that they have an adequate incentive to continue 
producing the standards?”)). The revenue to finance the important work of updating and revising 
the Standards as appropriate is generated solely through sales of the Standards. 

18. Due to the small membership size of Plaintiff NCME, and the relative minor 
portion of the membership of Plaintiffs AERA and APA who devote their careers to testing and 
assessment, it is highly unlikely that the members of the Sponsoring Organizations would vote 
for a dues increase to fund future revisions if sales revenue is lost by virtue of an entity like PRO 
allowing the Standards to be accessed on-line and downloaded or printed for free. As a result, 
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the Sponsoring Organizations would likely abandon their practice of periodically updating the 
Standards (Levine Deck, ]j 24; Camara Deck, f 24; Geisinger Deck, ]f 23; Ernesto Deck, ]j 33). 

19. As described above, the 1999 Standards are available for sale on-line through 
AERA. However, the 1999 Standards have been widely sold to professionals, academic 
institutions, libraries, professionals, students and academics. The 1999 Standards already in 
circulation are, therefore, available for review and consultation through those outlets. 

C. Why the Standards Are Published and Their Benefits 

20. The Standards are published “to promote the sound and ethical use of tests and to 
provide a basis for evaluating the quality of testing practices.” See 1999 Standards at 1 (quoted 
by the Court of Appeals, 896 F.3d at 441). The Standards set forth principles and guidelines, 
designed to provide a set of best practices to improve testing and assessment across multiple 
settings, including education and various areas of psychology. The Standards can and should be 
used in the sound and ethical development and use of tests, and also to evaluate the quality of 
tests and testing practices (Geisinger Deck, 18; Camara Deck, ]f 13; Wise Deck, ]j 12). 

21. The Standards are not simply intended for Plaintiffs’ members. They are intended 
for a broad audience that cuts across professions, backgrounds, and training. For example, they 
can guide test developers, sponsors, publishers, and users by providing criteria for the evaluation 
of tests, testing practices, and the effects of test use. Test user standards refer to those standards 
that help test users decide how to choose certain tests, interpret scores, or make decisions based 
on tests results. Test users include clinical or industrial psychologists, research directors, school 
psychologists, counselors, employment supervisors, teachers, and various administrators who 
select or interpret tests for their organizations. There is no mechanism, however, to enforce 
compliance with the Standards on the part of the test developer or test user (Camara Deck, ]J 14; 
Wise Deck, f 13; Geisinger Deck, ]f 19; Ernesto Deck, ]f 12). 
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22. The Standards promote the development of high quality tests and the sound use of 
results from such tests. Without such high quality standards, tests might produce scores that are 
not defensible or accurate, not an adequate reflection of the characteristic they were intended to 
measure, and not fair to the person tested. Thus, the Standards help ensure that measures of 
student achievement are relevant, that admissions decisions are fair, that employment hiring and 
professional credentialing results in qualified individuals being selected, and patients with 
psychological needs are diagnosed properly and treated accordingly. Quality tests protect the 
public from harmful decision-making and provide opportunities for education and employment 
that are fair to all who seek them (Camara Deck, 15; Wise Deck, U 14). 

23. The Standards apply broadly to a wide range of standardized vehicles and 
procedures that sample an individual’s behavior, including tests, assessments, inventories, scales, 
and other testing vehicles. The Standards apply equally to standardized multiple-choice tests, 
performance assessments (including tests comprised of only open-ended essays), and hands-on 
assessments or simulations. The main exceptions are that the Standards do not apply to 
unstandardized questionnaires ( e.g ., unstructured behavioral checklists or observational forms), 
teacher-made tests, and subjective decision processes (e.g., a teacher’s evaluation of students’ 
classroom participation over the course of a semester) (Camara Deck, ]j 16; Wise Deck, f 15; 
Geisinger Deck, H 20; Ernesto Deck, ]j 13). 

24. The Standards have been used to develop testing guidelines for such activities as 
college admissions, personnel selection, test translations, test user qualifications, and computer- 
based testing. The Standards also have been widely cited to address technical, professional, and 
operational norms for all forms of assessments that are professionally developed and used in a 
variety of settings. The Standards additionally provide a valuable public service to state and 
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federal governments that choose to use them. For instance, each testing company, when 
submitting proposals for testing administration, instead of relying on a patchwork of local, or 
even individual and proprietary, testing design and implementation criteria, may rely instead on 
the Standards to afford the best guidance for testing and assessment practices (Camara Deck, ]f 
17; Wise Deck, ]f 16; Geisinger Deck, 21; Ernesto Decl.,f 14). 

D. The Standards Are For Professional, Not Governmental, Use. 

25. As described above, the Standards are developed and designed for professional 
use. The Standards were not created or updated to address any governmental or regulatory need, 
nor in response to any legislative action or judicial decision. However, the Standards have been 
cited in judicial decisions related to the proper use and evidence for assessment, as well as by 
state and federal legislators. These citations in judicial decisions and during legislative 
deliberations occurred without any lobbying by the Plaintiffs (Levine Deck, ]f 12; Camara Deck, 
]f 18; Ernesto Deck, ]J 15; Wise Deck, f 17). 

26. During the first round of discovery in this case, APA located some 
correspondence or draft correspondence relating to APA’s support for legislation proposed in 
2001 by Senator Paul Wellstone (D-MN) on Fairness and Accuracy in High Stakes Educational 
Decisions for Students — a suggested amendment to the Elementary and Secondary Education 
Act (“No Child Left Behind Act”) 147 Cong. Rec. S. 4,644 (daily ed. May 9, 2001) (Ernesto 
Deck, TIT) 16-22, Exhs. NN-SS). Some of these letters are unsigned and are not printed on APA 
letterhead. Therefore, in accordance with APA practices, it is likely that the unsigned letters (not 
printed on letterhead) were internal discussion drafts that were never sent (Ernesto Deck, U 23). 

27. Regarding the signed letters printed on APA letterhead, they relate to Senator 
Wellstone’s proposed legislation mandating that tests and assessments administered by the states 
be of high quality and used appropriately for the benefit of test administrators and test takers. 
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These are goals that are consistent with APA policy as then reflected in the 1999 Standards. 
Even though Senator Wellstone’s amendments sought, in part, to mandate States’ compliance 
with the Standards, the Sponsoring Organizations had not actively advocated for this. In any 
event, Senator Wellstone’s proposed amendment never became law (Ernesto Decl., ]j 24, Exh. 
TT). 

28. APA’s search of its records did not disclose any further communications with 
Congress relating to the use of the Standards in legislation or proposed legislation, and, to the 
best of APA’s or the other Sponsoring Organizations’ knowledge, the Standards have not been 
referenced in legislation since 2001 (Ernesto Decl., U 25). Moreover, neither AERA nor NCME 
has ever communicated with Congress for the purpose of encouraging reference to the Standards 
in law (Levine Decl., TIT) 12-13; Wise Decl., U 18). None of the Plaintiffs has solicited any 
government agency to incorporate the Standards into the Code of Federal Regulations or other 
rules of federal or state agencies (Levine Decl., ]f 13; Ernesto Decl., U 26; Wise Decl., ]f 19). 

E. Plaintiffs’ Development of the 1999 Edition of the Standards 

29. Preparing and revising the Standards entails intensive labor and considerable 
cross-disciplinary expertise. Each time the Standards are revised, the Sponsoring Organizations 
select and arrange for meetings of the Joint Committee, composed of leading authorities in 
psychological and educational assessments. During these meetings, certain individual standards 
are combined, pared down, or augmented, while others are deleted altogether, and some are 
created as whole new individual standards. 

30. The 1999 Standards took more than five years to complete. It is the result of 
work put in by the Joint Committee to generate a set of best practices on educational and 
psychological testing that are respected and relied upon by leaders in their fields (Levine Decl., f 
11; Camara Decl., f 11; Wise Decl., ]j 11). Draft revisions of the 1985 Standards, which became 
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the 1999 Standards, were widely distributed for public review and comment three times during 
this revision effort to gauge whether the testing community believed the revised drafts to be 
current and inclusive of the topics at issue (Schneider Decl., U 6). The Joint Committee received 
thousands of pages of comments and proposed text revisions from Plaintiffs’ members; 
scientific, professional, trade, and advocacy groups; credentialing boards; state and federal 
government agencies; test publishers and developers; and academic institutions. While the Joint 
Committee reviewed and took under advisement these comments, the final language of the 1999 
Standards was produced by the Joint Committee members (Camara Deck, f 12; Schneider Deck, 
]j 7). More than half the content of the 1999 Standards resulted from newly written prose of the 
Joint Committee (Camara Deck, f 12). 

F. Description of the 1999 Standards (Hutter Decl. Exhibit 1) 

31. Cover-to-cover, the 1999 Standards are 205 pages long. The vast bulk of those 
205 pages is explanatory and background material and commentary, not the black-letter 
standards themselves. For instance, the 1999 Standards include: 

• A front cover, a back cover, a cover page, and a page showing copyright and 
publisher infonnation; 

• A table of contents showing how the 1999 Standards are organized; 

• A preface providing a short history of the Standards and identifying the people 
and organizations that participated in creating the 1999 Standards; 

• A five-plus page Introduction that gives an overview of the testing process; 
introduces the 1999 Standards; offers some cautions to be exercised in using 
the Standards; explains some updates specific to the 1999 edition of the 
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Standards; clarities how the 1999 Standards use the term “construct”; and 
elaborates on how the 1999 Standards are organized; 

• An index pointing users to the pages in the 1999 Standards that refer to 
specific tenns; and 

• A glossary defining how certain terms are specifically used in the 1999 
Standards. 

32. The remaining 164 pages of the 1999 Standards consist of three substantive parts. 
Part I is titled “Test Construction, Evaluation, and Documentation.” Part II is titled “Fairness in 
Testing.” Part III is titled “Testing Applications.” Each of the three parts is broken up into 
chapters. Part I, for example, contains chapters on validity; reliability and errors of 
measurement; test development and revision; scaling, norming, and score comparability; test 
administration, scoring, and reporting; and supporting documentation for tests. 

33. Each chapter in the 1999 Standards consists, in turn, of two overall sections. First 
there is a section of background and explanatory material organized under various topical 
subheadings. As the introduction to the 1999 Standards makes clear, while the section of 
background material opening each chapter may be helpful to understanding the black-letter 
standards, “it should not be interpreted as imposing additional standards.” 

34. The background material at the outset of each chapter is followed in each chapter 
by a section that includes the individual, black-letter standards themselves. Each black-letter 
standard is stated in bold-face type, one paragraph long, and is written in a prescriptive manner. 
Each individual standard is followed by a Comment on which may help explain, elaborate on or 
offer examples. The comments are designed to assist in applying the black-letter standards, but 
they do not impose additional requirements beyond the black-letter standards. Even in the part 
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of each chapter that contains the standards (which is less than half the chapter), more words tend 
to be devoted to the commentary than to the black letter standards themselves. 

35. The 1999 Standards, including all chapters, contains 272 standards in total. The 
standards take up less than 15% of 1999 Standards (as detennined by comparing the number of 
lines of text devoted to standards in relation to the lines in the text overall). See Hutter Deck ]f 6. 

G. The Federal Practice of Incorporation by Reference. 

36. The National Technology Transfer and Advancement Act of 1995 (“NTTAA”) 
requires federal agencies to use privately-developed standards to achieve federal objectives 
whenever possible. 2 Pub. L. No. 104-113 § 12, 110 Stat. 775, 782-83 (1996), codified at 15 
U.S.C. § 272. Specifically it declares that “all Federal agencies and departments shall use 
technical standards that are developed or adopted by voluntary consensus bodies, using such 
technical standards as a means to carry out policy objectives or activities.” Id. 

37. One method through which federal agencies avail themselves of privately- 
developed standards is by incorporating the standards by reference. The Code of Federal 
Regulations currently contains more than 23,000 incorporations by reference, though not all of 
the material incorporated are standards developed by private standards-setting bodies. See 


The NTTAA was the product of a study of the system of privately-developed consensus standards as it had 
evolved in the United States, which emerged with the recommendation that government agencies continue to avail 
itself of that system in aid of its own mission. 

Specifically, in 1991 Congress enacted P.L. 102-245, requesting the National Research Council study 
standards development. See National Research Council, Standards, Conformity Assessment, and Trade into the 21st 
Century vii (National Academy Press 1995), available at http://www.nap.edU/read/4921/chapter/l. That study’s 
overview of the U.S. standards-development system noted that many standards developers “offset expenses and 
generate income through sales of standards documents, to which they hold the copyright. For many SDOs, 
publishing is a significant source of operating revenue.” Id. at 32 (emphasis added). The study concluded that the 
“U.S. standards development system serves the national interest well” by “supporting] efficient and timely 
development of product and process standards that meet economic and public interests.” Id. at 157. The study 
recommended that Congress pass a law to promote the use of privately developed voluntary consensus standards by 
federal agencies. Congress responded by enacting the NTTAA, codified at 15 U.S.C. § 272. 
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Bremer, Teaching Guide: Incorporation by Reference, 2019 Administrative Law Review at 321; 
Bremer, Private Standards in Public Law, 63 Kansas L. Rev. 279. 

38. The Office of Management and Budget has explained that incorporation by 
reference (i) saves the government the cost of developing standards on its own; (ii) provides 
incentives to establish standards serving national needs; (iii) promotes efficiency and economic 
competition through harmonized standards; and (iv) furthers the federal policy of relying on the 
private sector to meet government needs for goods and services. OMB Circular No. A-l 19, 63 
Fed. Reg. 8546 (Revised Feb. 10, 1998). See also Final Revision of OMB Circular A-l 19, 81 
Fed. Reg. 4673 (Jan. 27, 2016). 

39. OMB has recognized that it is essential to not interfere with the ability of the 
standard-setting organizations to charge for the use of their works: “If we required that all 
materials IBR’d into the CFR be available for free, that requirement would compromise the 
ability of regulators to rely on voluntary consensus standards, possibly requiring them to create 
their own standards, which is contrary to the NTTAA and the OMB Circular A-l 19.” See 
Incorporation by Reference, Announcement of Final Rule, Office of the Federal Register, 79 
Fed. Reg. 66267, 66268 (Nov. 7, 2014), available at https://federalregister.gOv/a/2014-26445 . 

40. The practice of incorporation by reference is currently codified at 5 U.S.C. 

§552(a)(1), which clarifies that no material incorporated by reference can be deemed binding - 

in the sense of adversely affecting any person - unless the material is reasonably available to the 

class of affected persons. Thus, any standard incorporated by reference is self-limiting in its 

application if not “reasonably available.” 

Except to the extent that a person has actual and timely notice of 
the terms thereof, a person may not in any manner be required to 
resort to, or be adversely affected by, a matter required to be 
published in the Federal Register and not so published. For 
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purposes of this paragraph, matter reasonably available to the 
class of persons affected thereby is deemed published in the 
Federal Register when incorporated by reference therein with the 
approval of the Director of the Federal Register. 

41. The Office of Federal Register in turn has promulgated regulations governing the 
process for agencies to follow in order to obtain approval to incorporate materials by reference in 
the CFR. 1 C.F.R. §51 (2014). Those requirements interpret and implement the statutory 
mandate that, in order for matters set forth in the register to be binding, they must be “reasonably 
available to the class of persons affected.” 5 U.S.C. § 552(a)(1); 1 C.F.R. § 51.7(a)(3). They 
require that (i) a copy of the incorporated material must be on tile with the Office of the Federal 
Register and (ii) that the regulations incorporating such material must state the ways those 
incorporated materials are reasonably available to interested parties. 1 C.F.R. §§51.3, 51.5. 
There is no requirement that such materials be available to the public at no cost, or on the 
internet 

H. DOE’s Incorporation By Reference of Some Parts of the 1999 Standards 

42. Plaintiffs learned primarily during the course of these proceedings that the 
Department of Education has incorporated certain standards set forth in the 1999 Standards in 
three sections of the Code of Federal Regulations. As seen below, none of these provisions 
apply to conventional primary conduct or purport to apply mandatory enforceable rules with 
penalties attached. Rather, they address how tests may qualify for purposes relevant to 
Department of Education grant programs. 

43. 34 C.F.R. §668.146(b)(6) is the only regulation that was cited by PRO as the basis 
for its incorporation-by-reference rationale for posting the 1999 Standards. That regulation is 
part of a subpart of regulations setting forth provisions under which a student without a high 
school diploma or its recognized equivalent may become eligible to receive funds under Title IV 
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of the Higher Education Act. The student may become eligible by obtaining a diploma or its 
equivalent. Another way for a student to become eligible for Title IV funds is to pass a test that 
satisfies certain criteria including “all standards for test construction provided in the 1999 edition 
of the Standards for Educational and Psychological Testing, prepared by a joint committee of the 
American Educational Research Association, the American Psychological Association, and the 
National Council on Measurement in Education incorporated by reference in this section.” The 
regulation then goes on to note that the incorporation by reference was properly approved, 
identifies where the material referenced (the 1999 Standards) is on file, and provides a phone 
number and website for the National Archives and Records Administration “[f]or infonnation on 
the availability of this material at NARA.” The regulation also identifies the AERA website, 
noting that the document may also be obtained from there. 

44. The individual standards incorporated by reference in 34 C.F.R. 668.146(b)(6) are 
set forth in Part I of the 1999 Standards, entitled “Test Construction, Evaluation and 
Documentation.” Part I actually consists of six chapters, covering Validity, Reliability and 
Errors of Measurement, Test Development and Revision, Scales, Norms and Score 
Comparability, Test Administration, Scoring, and Reporting, and Supporting Documentation for 
Tests. Even assuming that all six chapters of Part I are being referenced by the regulation, the 
actual bold-face “standards” set forth in these chapters are very limited. 

45. For example, the chapter within Part I on “validity” runs from pp. 9-24, begins 
with background and explanatory material, and contains standards only on pages 17-24 
(Standards 1.1-1.24), including comments on each standard. The comments on each standard are 
typically as long as, or longer than the standards themselves. 
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46. The chapter within Part I on Reliability and Errors is eleven pages long and 
begins with 6 pages of background and discussion of various topics and considerations. 
Standards (2.1-2.20) are set forth on 6 pages, and the comments to the standards are again longer 
than the referenced standards themselves. 

47. The chapter on Test Development and Revision runs from pages 37-48 and the 
first 6 pages describe background and discussions of various related topics and considerations. 
The actual standards are set forth within pages 43-48 (standards 3.1- 3.27), and again, most of 
that appears to be commentary, rather than the individual standards themselves (though some of 
the standards have no immediately following “comment” and some comments are as brief as the 
standard). 

48. The next three chapters in Part I follow a similar format. 

49. 34 C.F.R. §668.148(a)(l)(iv) is also a part of the subpart of the regulations setting 
forth provisions under which a student without a high school diploma or its recognized 
equivalent may become eligible to receive funds under Title IV. This provision covers tests 
developed for non-native speakers of English enrolled in a program taught in their native 
language. Such tests must, according to this provision, be “[djeveloped in accordance with 
guidelines provided in the “’Testing Individuals of Diverse Linguistic Backgrounds’ section of 
the” 1999 Standards. 

50. That chapter of the 1999 Standards runs from pages 91 to 100, with the first 6 
pages covering background and explanatory topics. The actual standards are found at pages 97 
to 100, and again, the actual standards themselves appear to occupy less than half of those pages, 
with comments taking up more space. 
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51. 34 C.F.R. §462.13(c)(1) and (f)(1) are part of the regulations establishing a 
process for reviewing the suitability of tests, as submitted by test publishers, for use in the 
National Reporting System for Adult Education, which is an outcome-based accountability 
system for evaluating state-administered, federally-funded adult education programs. Among the 
criteria used to detennine whether a test is suitable for use in the that system is that “[t]he test 
must meet all applicable and feasible standards for test construction and validity provided in the 
1999 edition of the Standards[.]” But a test or publisher that seeks to qualify under this program 
is specifically offered the opportunity to demonstrate why a particular standard is not feasible or 
applicable. 

52. This regulation, like 34 C.F.R. §668.146(b)(6), references Parti of the 1999 
Standards. The regulation itself contains similar information and directions as to how and where 
to find the incorporated standards. 

53. In addition, subsection (f)(1) provides that “For a test that has been modified for 
individuals with disabilities, the Error! Hyperlink reference not valid.must,” among other 
things, “[pjrovide documentation that it followed the guidelines provided in the Testing 
Individuals With Disabilities section of the 1999 edition of the Standards for Educational and 
Psychological Testing. ” That is reference to a different chapter of the 1999 Standards, contained 
in Part 2. That chapter comprises 8 pages, the first 5 of which provide background and 
explanatory material, which is followed by 3 pages containing the black letter standards (10.1- 
10.12), many of which are followed a Comment. 

54. Less than 6 per cent of the 1999 Standards is occupied by the standards 
incorporated by reference in the three regulation (as determined by comparing the number of 
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lines of text in the referenced standards with the number of lines in the text overall). Hutter 
Decl. H 6. 

55. Plaintiffs are not aware that DOE has ever had occasion to enforce the provisions 
of its regulations referencing the 1999 Standards. 

56. In addition, Plaintiffs are not specifically aware of tests developed specifically to 
meet these DOE regulations. Rather, tests meeting these criteria may have been developed 
entirely independent of the regulations because high quality tests generally would, as a matter of 
basic professionalism, be developed in a manner consistent with the Standards, but without 
regard to the fact that some of the standards are cited in DOE regulations. The exception could, 
of course, be for 34 C.F.R. §462.13 (f)(1), which requires the test publisher to provide 
documentation that it affirmatively meets the requirements of the referenced portion of the 1999 
Standards. But plaintiffs are not in possession of any information regarding how or whether that 
provision has been applied. 

57. Copies of the 1999 Standards are available, as identified in the CFR, in the 
reading room and from AERA, either by ordering on-line or over the telephone. In addition, 
existing copies of the 1999 Standards are available and in circulation throughout academia 
(including schools of education and departments of psychology), are in the possession of 
organizations involved in testing, can be accessed in public and university libraries, and would 
expectably be accessible and referenced in connection with the development and use of tests 
irrespective of whether at some point the test would be in any way subject to the DOE 
regulations. 
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I. Defendants Copied the 1999 Standards In Toto and Verbatim, and Enabled 
Others to Copy Them, Without Plaintiffs’ Permission 

58. Defendant Public.Resource.Org (PRO) is a California non-profit corporation 
founded in 2007 by Mr. Carl Malamud, with the explicit aim of making government information 
more accessible, with particular emphasis on the law (Hudis Decl, ]j 2, Exh. A, pp. 77, 93-94, 
163-164). The identified purpose and objective of PRO is to create and maintain so-called 
informational “public works projects for the Internet” (Hudis Deck, ]f 2, Exh. A, pp. 94-95, 105— 
09, ]j 3, Exh. B, Section II.B., ]f 4, Exh. C. Section 2.1). 

59. PRO creates digital copies of standards referenced or incorporated in regulations, 
without the permission of the copyright owners. It does so as a matter of principle. That is, PRO 
does not purport to identify any specific need or reason for such copying. Rather, PRO simply 
believes, and acts on the belief, that all such material should copied and made available for 
copying on the internet. Its mission includes the broad and indiscriminate objective “to make the 
law and other government materials more widely available.” 896 F.3d at 444 (quoting 
Malamud’s Declaration in this case). 

60. In March 2012, PRO began copying standards incorporated by reference into the 
Code of Federal Regulations, and facilitating copying by others. In May 2012, PRO began the 
process of posting copies these standards to its website. “Between 2012 and 2014, PRO uploaded 
hundreds of technical standards, which, collectively, were downloaded tens of thousands of 
times.” Id. 

61. Specifically, on May 17, 2012, PRO bought a used hard copy of the 1999 

Standards from an Amazon re-seller (Hudis Deck, ]j 2, Exh. A, pp. 232-240, f 21, Exh. T, Int. 
Ans. 1, 22-23, Exh. U). Upon receipt of the purchased paper copy, Malamud disassembled 

the book, removed the spine, trimmed the pages to give them an even border, scanned the pages 
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to create a PDF, and named the PDF file “aera.standards.l999.pdf.” Malamud then appended a 
cover sheet, a self-made “Certificate,” to the front of the PDF file giving a false semblance of 
governmental imprimatur to the unauthorized copying and online posting of the 1999 Standards 
(Hudis Decl., ]J 2, Exh. A, pp. 257-59, 261-64, 21, Exh. T, Int. Ans. 3-4, H 26. Malamud did 
nothing else to modify or transform the PDF file or the Standards. He simply copied and posted 
the 1999 Standards, in its entirety, to his company’s website. He did not provide for word¬ 
searching, online identification, or text-to-speech utilization for the blind and visually impaired 
(Hudis Decl., 1 27, Exh. Z, pp. 30, 122, 200-01, 206, 271-72, 315-16). PRO published the 
infringing digital copy of the 1999 Standards on a website titled https://law.resource.org. 

62. The Internet Archive is a nonprofit organization whose mission is to build and 
maintain a digital library of the Internet. The Internet Archive builds this internet library — 
which it makes available for public use — by scanning, digitally capturing, and saving 
electronically scanned and captured third-party websites, and by receiving submissions from 
third parties who have user accounts enabling them to upload content (Hudis Deck, ]f 29, Exh. 
BB, pp. 31-41). Malamud has such user account access (Hudis Deck, ]J 29, Exh. BB, pp. 51 — 
56), and he uploaded the entirety of the 1999 Standards to the Internet Archive’s website on May 
26-27, 2012 (Hudis Deck, U 29, Exh. BB, pp. 59-112, f 30, Exh. CC 3-18 therein), H 32, 
Exh. EE, ]| 33). PRO posted Plaintiffs’ 1999 Standards to its website and the Internet Archive 
website without the pennission or authorization of any of the Sponsoring Organizations (Hudis 
Deck, ]J 35, Exh. HH, Admission Nos. 4-5; Levine Deck, ]f 29; Ernesto Deck, ]f 35; Wise Deck, 
126). 

63. The copy of the Sponsoring Organizations’ 1999 Standards that Malamud 
published to the Internet Archive at https://archive.org/details/gov.law.aera.standards.1999 was 
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in the same format, using the same cover sheet or “Certificate” employed by PRO in the posting 
of the Sponsoring Organizations’ 1999 Standards to Defendant’s own website. All of the 
surrounding text associated with the posting to the Internet Archive website was inserted by 
Malamud — including the insertion of “Creative Commons License: CCO 1.0 Universal ,” 
indicating that no rights are being asserted over the item (Hudis DecL, ]j 2, Exh. A, pp. 275-84, ]f 
29, Exh. BB, pp. 57-63, If 30, Exh. CC (12 therein), ]f 34, Exh. GG): 

64. Although based on incomplete reporting (Hudis DecL, ]J 2, Exh. A, pp. 272-74, 
328-36), during the near two-year period that the Sponsoring Organizations’ 1999 Standards 
were first posted on PRO’s https://law.resource.org website, they were accessed at least 4,164 
times (Hudis DecL, H 21, Exh. T, Int. Ans. 2 and Amended Ans. 5 (labeled 6)). During that same 
period, the Sponsoring Organizations’ 1999 Standards were accessed on the Internet Archive 
https://archive.org w ebsite 1,290 times (Hudis DecL, If 29, Exh. BB, pp. 124-132, ]f 37, Exh. II). 

65. The Internet Archive’s website is open to the public and does not restrict an 
Internet user’s ability to download or print the Sponsoring Organizations’ 1999 Standards. PRO 
also placed no such restrictions on its website (Hudis DecL, ]f 2, Exh. A, pp. 347-48). There 
were no sign-up procedures to enter PRO’s https://law.resource.org website, nor was there any 
Digital Rights Management (or “DRM”) plan to protect against, or identify, further copying of 
the files accessed from PRO’s site (Hudis DecL, If 27, Exh. Z, pp. 324-27, 167-73). 

66. There is no way for Plaintiffs to calculate the number of university/college 
professors, students, testing companies and others who would have purchased Plaintiffs’ 
Standards but for their wholesale posting on PRO’s https://law.resource.org w ebsite and the 
Internet Archive http://archive.org website (Levine DecL, If 30; Geisinger DecL, If 24). 
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67. In late 2013 and early 2014, the Sponsoring Organizations became aware that the 
1999 Standards had been posted on the Internet without their authorization, and that students 
were obtaining free copies from the posting source. Upon further investigation, they discovered 
that PRO was the source of the online posting (Camara Decl., 21, Exh. MMM; Wise Decl., TIT) 
27-28, Exh. LLL). 

68. In December 2013, Plaintiff AERA requested in writing that PRO remove the 
1999 Standards from its online postings (Levine Decl., 31, Exh. UUU). Defendant refused 
(Hudis Decl., ]f 2, Exh. A, pp. 310-19, U 38, Exh. JJ, 1} 39, Exh. KK). Once this lawsuit was fded 
and the Sponsoring Organizations threatened to file a motion for a preliminary injunction, PRO 
agreed in June 2014 to remove its postings of the 1999 Standards from its 
https://law.resource.org w ebsite and from Internet Archive’s https://archive.org website, pending 
a resolution of this litigation on the merits. PRO’S undertaking included the promise not to post 
any revision of the 1999 Standards (i.e., the 2014 Standards) pending the outcome of this 
litigation on the merits (Hudis Deck, ]j2, Exh. A, pp. 322-28, ]J 40, Exh. LL, 41, Exh. MM). 

69. Notwithstanding that undertaking, it is currently posted on the Internet Archive 

website, stating that it was uploaded on May 26, 2012. That posting includes the full volume of 
the 1999 Standards, verbatim, covered with the self-created “Certificate” stating “By Authority 
of the United States of America ... Legally Binding Document,” and stating that the document is 
posted by Public.Resource.Org. Inc. The website cites a single section of the CFR, 34 C.F.R. 
§668.148(a)(2)(iv). The website posting provides a variety of means by which the 1999 
Standards can be downloaded. See Hutter Decl. Exh 3. The website reflected that as of 
September 11, 2019, it was viewed 1445 times. See 

https://archive.org/details/gov.law.aera.standards.1999/page/nl 
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J. This Litigation, and the Court of Appeals’ Remand Direction 

70. In 2017, this Court granted Plaintiffs’ motion for summary judgment on their 
copyright claim. The Court held that Plaintiffs own the copyright to the 1999 Standards; that 
Plaintiffs’ copyright is valid; that nothing about the 1999 Standards’ incorporation into federal 
law negated the copyright; that PRO’S posting on the internet was infringing; and that PRO’S 
wholesale copying and posting of the entirety of the 1999 Standards to the internet, from where it 
could be downloaded, was not fair use. Regarding the fair use issue, this Court’s decision 
recognized that PRO had the burden on that issue, and the Court addressed the specific 
arguments that PRO had made - arguments that PRO advanced across the board, without 
addressing the differences between the many and varied standards at issue in the ASTM case, or 
the differences between how those standards were referenced in federal regulations. Based on 
those holdings, the Court enjoined PRO from continuing to post the 1999 Standards. 

71. In 2018, the Court of Appeals for the District of Columbia Circuit vacated this 
Court’s summary judgment and injunction order and remanded the case for further proceedings. 
The Court of Appeals’ remand order was limited to a single issue: fair use. The Court of 
Appeals noted that PRO had offered an “undifferentiated” theory of why its publication of any 
standards incorporated by reference in federal regulations qualified as fair use (or should be 
denied copyright protection at all). This Court had therefore addressed those arguments of PRO 
in the undifferentiated tenns under which they were presented. 

72. But the Court of Appeals believed that approach “failed to account for the 
variation among the standards at issue and ... failed to consider each fair use claim ‘on its own 
facts.’” Thus, it noted the many and varied standards and incorporations by reference at issue (at 
least in the ASTM case), and directed that: 
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On remand, the district court will need to develop a fuller record 
regarding the nature of each of the standards at issue, the way in 
which they are incorporated, and the manner and extent to which 
they were copied by PRO in order to resolve this “mixed question 
of law and fact.” 

73. In light of that direction, this Court allowed PRO an extended period to take 
additional discovery, which it did. Plaintiffs did not take any additional discovery. 


Respectfully submitted, 
s/ Clifton S. Elgarten 

John I. Stewart, Jr. (D.C. Bar No. 913905) 
Clifton S. Elgarten (D.C. Bar No. 366898) 
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